{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "intro_to_neural_nets.ipynb",
      "version": "0.3.2",
      "provenance": [],
      "collapsed_sections": [
        "O2q5RRCKqYaU",
        "vvT2jDWjrKew",
        "copyright-notice"
      ]
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "accelerator": "TPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "copyright-notice"
      },
      "source": [
        "#### Copyright 2017 Google LLC."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "copyright-notice2",
        "cellView": "both",
        "colab": {}
      },
      "source": [
        "# Licensed under the Apache License, Version 2.0 (the \"License\");\n",
        "# you may not use this file except in compliance with the License.\n",
        "# You may obtain a copy of the License at\n",
        "#\n",
        "# https://www.apache.org/licenses/LICENSE-2.0\n",
        "#\n",
        "# Unless required by applicable law or agreed to in writing, software\n",
        "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
        "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
        "# See the License for the specific language governing permissions and\n",
        "# limitations under the License."
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "eV16J6oUY-HN",
        "colab_type": "text",
        "slideshow": {
          "slide_type": "slide"
        }
      },
      "source": [
        " # 神经网络简介"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_wIcUFLSKNdx",
        "colab_type": "text"
      },
      "source": [
        " **学习目标：**\n",
        "  * 使用 TensorFlow `DNNRegressor` 类定义神经网络 (NN) 及其隐藏层\n",
        "  * 训练神经网络学习数据集中的非线性规律，并实现比线性回归模型更好的效果"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_ZZ7f7prKNdy",
        "colab_type": "text"
      },
      "source": [
        " 在之前的练习中，我们使用合成特征来帮助模型学习非线性规律。\n",
        "\n",
        "一组重要的非线性关系是纬度和经度的关系，但也可能存在其他非线性关系。\n",
        "\n",
        "现在我们从之前练习中的逻辑回归任务回到标准的（线性）回归任务。也就是说，我们将直接预测 `median_house_value`。"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "J2kqX6VZTHUy",
        "colab_type": "text"
      },
      "source": [
        " ## 设置\n",
        "\n",
        "首先加载和准备数据。"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "AGOM1TUiKNdz",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "from __future__ import print_function\n",
        "\n",
        "import math\n",
        "\n",
        "from IPython import display\n",
        "from matplotlib import cm\n",
        "from matplotlib import gridspec\n",
        "from matplotlib import pyplot as plt\n",
        "import numpy as np\n",
        "import pandas as pd\n",
        "from sklearn import metrics\n",
        "import tensorflow as tf\n",
        "from tensorflow.python.data import Dataset\n",
        "\n",
        "tf.logging.set_verbosity(tf.logging.ERROR)\n",
        "pd.options.display.max_rows = 10\n",
        "pd.options.display.float_format = '{:.1f}'.format\n",
        "\n",
        "california_housing_dataframe = pd.read_csv(\"https://download.mlcc.google.cn/mledu-datasets/california_housing_train.csv\", sep=\",\")\n",
        "\n",
        "california_housing_dataframe = california_housing_dataframe.reindex(\n",
        "    np.random.permutation(california_housing_dataframe.index))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "2I8E2qhyKNd4",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def preprocess_features(california_housing_dataframe):\n",
        "  \"\"\"Prepares input features from California housing data set.\n",
        "\n",
        "  Args:\n",
        "    california_housing_dataframe: A Pandas DataFrame expected to contain data\n",
        "      from the California housing data set.\n",
        "  Returns:\n",
        "    A DataFrame that contains the features to be used for the model, including\n",
        "    synthetic features.\n",
        "  \"\"\"\n",
        "  selected_features = california_housing_dataframe[\n",
        "    [\"latitude\",\n",
        "     \"longitude\",\n",
        "     \"housing_median_age\",\n",
        "     \"total_rooms\",\n",
        "     \"total_bedrooms\",\n",
        "     \"population\",\n",
        "     \"households\",\n",
        "     \"median_income\"]]\n",
        "  processed_features = selected_features.copy()\n",
        "  # Create a synthetic feature.\n",
        "  processed_features[\"rooms_per_person\"] = (\n",
        "    california_housing_dataframe[\"total_rooms\"] /\n",
        "    california_housing_dataframe[\"population\"])\n",
        "  return processed_features\n",
        "\n",
        "def preprocess_targets(california_housing_dataframe):\n",
        "  \"\"\"Prepares target features (i.e., labels) from California housing data set.\n",
        "\n",
        "  Args:\n",
        "    california_housing_dataframe: A Pandas DataFrame expected to contain data\n",
        "      from the California housing data set.\n",
        "  Returns:\n",
        "    A DataFrame that contains the target feature.\n",
        "  \"\"\"\n",
        "  output_targets = pd.DataFrame()\n",
        "  # Scale the target to be in units of thousands of dollars.\n",
        "  output_targets[\"median_house_value\"] = (\n",
        "    california_housing_dataframe[\"median_house_value\"] / 1000.0)\n",
        "  return output_targets"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "pQzcj2B1T5dA",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1209
        },
        "outputId": "8f4bf430-0fca-4b34-f87c-55e60d6bb453"
      },
      "source": [
        "# Choose the first 12000 (out of 17000) examples for training.\n",
        "training_examples = preprocess_features(california_housing_dataframe.head(12000))\n",
        "training_targets = preprocess_targets(california_housing_dataframe.head(12000))\n",
        "\n",
        "# Choose the last 5000 (out of 17000) examples for validation.\n",
        "validation_examples = preprocess_features(california_housing_dataframe.tail(5000))\n",
        "validation_targets = preprocess_targets(california_housing_dataframe.tail(5000))\n",
        "\n",
        "# Double-check that we've done the right thing.\n",
        "print(\"Training examples summary:\")\n",
        "display.display(training_examples.describe())\n",
        "print(\"Validation examples summary:\")\n",
        "display.display(validation_examples.describe())\n",
        "\n",
        "print(\"Training targets summary:\")\n",
        "display.display(training_targets.describe())\n",
        "print(\"Validation targets summary:\")\n",
        "display.display(validation_targets.describe())"
      ],
      "execution_count": 3,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Training examples summary:\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "display_data",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>latitude</th>\n",
              "      <th>longitude</th>\n",
              "      <th>housing_median_age</th>\n",
              "      <th>total_rooms</th>\n",
              "      <th>total_bedrooms</th>\n",
              "      <th>population</th>\n",
              "      <th>households</th>\n",
              "      <th>median_income</th>\n",
              "      <th>rooms_per_person</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>count</th>\n",
              "      <td>12000.0</td>\n",
              "      <td>12000.0</td>\n",
              "      <td>12000.0</td>\n",
              "      <td>12000.0</td>\n",
              "      <td>12000.0</td>\n",
              "      <td>12000.0</td>\n",
              "      <td>12000.0</td>\n",
              "      <td>12000.0</td>\n",
              "      <td>12000.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>mean</th>\n",
              "      <td>35.6</td>\n",
              "      <td>-119.6</td>\n",
              "      <td>28.6</td>\n",
              "      <td>2616.9</td>\n",
              "      <td>535.1</td>\n",
              "      <td>1416.8</td>\n",
              "      <td>496.8</td>\n",
              "      <td>3.9</td>\n",
              "      <td>2.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>std</th>\n",
              "      <td>2.1</td>\n",
              "      <td>2.0</td>\n",
              "      <td>12.6</td>\n",
              "      <td>2153.0</td>\n",
              "      <td>415.9</td>\n",
              "      <td>1093.1</td>\n",
              "      <td>377.7</td>\n",
              "      <td>1.9</td>\n",
              "      <td>1.2</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>min</th>\n",
              "      <td>32.5</td>\n",
              "      <td>-124.3</td>\n",
              "      <td>1.0</td>\n",
              "      <td>11.0</td>\n",
              "      <td>3.0</td>\n",
              "      <td>8.0</td>\n",
              "      <td>2.0</td>\n",
              "      <td>0.5</td>\n",
              "      <td>0.1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>25%</th>\n",
              "      <td>33.9</td>\n",
              "      <td>-121.8</td>\n",
              "      <td>18.0</td>\n",
              "      <td>1453.0</td>\n",
              "      <td>295.0</td>\n",
              "      <td>788.0</td>\n",
              "      <td>280.0</td>\n",
              "      <td>2.6</td>\n",
              "      <td>1.5</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>50%</th>\n",
              "      <td>34.2</td>\n",
              "      <td>-118.5</td>\n",
              "      <td>29.0</td>\n",
              "      <td>2110.0</td>\n",
              "      <td>431.0</td>\n",
              "      <td>1162.0</td>\n",
              "      <td>406.0</td>\n",
              "      <td>3.5</td>\n",
              "      <td>1.9</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>75%</th>\n",
              "      <td>37.7</td>\n",
              "      <td>-118.0</td>\n",
              "      <td>37.0</td>\n",
              "      <td>3136.0</td>\n",
              "      <td>648.0</td>\n",
              "      <td>1712.0</td>\n",
              "      <td>603.0</td>\n",
              "      <td>4.8</td>\n",
              "      <td>2.3</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>max</th>\n",
              "      <td>42.0</td>\n",
              "      <td>-114.3</td>\n",
              "      <td>52.0</td>\n",
              "      <td>37937.0</td>\n",
              "      <td>5471.0</td>\n",
              "      <td>16122.0</td>\n",
              "      <td>5189.0</td>\n",
              "      <td>15.0</td>\n",
              "      <td>55.2</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "       latitude  longitude  ...  median_income  rooms_per_person\n",
              "count   12000.0    12000.0  ...        12000.0           12000.0\n",
              "mean       35.6     -119.6  ...            3.9               2.0\n",
              "std         2.1        2.0  ...            1.9               1.2\n",
              "min        32.5     -124.3  ...            0.5               0.1\n",
              "25%        33.9     -121.8  ...            2.6               1.5\n",
              "50%        34.2     -118.5  ...            3.5               1.9\n",
              "75%        37.7     -118.0  ...            4.8               2.3\n",
              "max        42.0     -114.3  ...           15.0              55.2\n",
              "\n",
              "[8 rows x 9 columns]"
            ]
          },
          "metadata": {
            "tags": []
          }
        },
        {
          "output_type": "stream",
          "text": [
            "Validation examples summary:\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "display_data",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>latitude</th>\n",
              "      <th>longitude</th>\n",
              "      <th>housing_median_age</th>\n",
              "      <th>total_rooms</th>\n",
              "      <th>total_bedrooms</th>\n",
              "      <th>population</th>\n",
              "      <th>households</th>\n",
              "      <th>median_income</th>\n",
              "      <th>rooms_per_person</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>count</th>\n",
              "      <td>5000.0</td>\n",
              "      <td>5000.0</td>\n",
              "      <td>5000.0</td>\n",
              "      <td>5000.0</td>\n",
              "      <td>5000.0</td>\n",
              "      <td>5000.0</td>\n",
              "      <td>5000.0</td>\n",
              "      <td>5000.0</td>\n",
              "      <td>5000.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>mean</th>\n",
              "      <td>35.6</td>\n",
              "      <td>-119.6</td>\n",
              "      <td>28.4</td>\n",
              "      <td>2707.9</td>\n",
              "      <td>549.7</td>\n",
              "      <td>1460.2</td>\n",
              "      <td>511.9</td>\n",
              "      <td>3.9</td>\n",
              "      <td>2.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>std</th>\n",
              "      <td>2.1</td>\n",
              "      <td>2.0</td>\n",
              "      <td>12.7</td>\n",
              "      <td>2242.1</td>\n",
              "      <td>434.4</td>\n",
              "      <td>1269.3</td>\n",
              "      <td>400.4</td>\n",
              "      <td>1.9</td>\n",
              "      <td>1.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>min</th>\n",
              "      <td>32.6</td>\n",
              "      <td>-124.3</td>\n",
              "      <td>1.0</td>\n",
              "      <td>2.0</td>\n",
              "      <td>1.0</td>\n",
              "      <td>3.0</td>\n",
              "      <td>1.0</td>\n",
              "      <td>0.5</td>\n",
              "      <td>0.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>25%</th>\n",
              "      <td>33.9</td>\n",
              "      <td>-121.8</td>\n",
              "      <td>18.0</td>\n",
              "      <td>1475.8</td>\n",
              "      <td>301.0</td>\n",
              "      <td>794.8</td>\n",
              "      <td>284.0</td>\n",
              "      <td>2.6</td>\n",
              "      <td>1.5</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>50%</th>\n",
              "      <td>34.3</td>\n",
              "      <td>-118.5</td>\n",
              "      <td>28.0</td>\n",
              "      <td>2178.0</td>\n",
              "      <td>441.5</td>\n",
              "      <td>1179.5</td>\n",
              "      <td>417.0</td>\n",
              "      <td>3.6</td>\n",
              "      <td>2.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>75%</th>\n",
              "      <td>37.7</td>\n",
              "      <td>-118.0</td>\n",
              "      <td>37.0</td>\n",
              "      <td>3185.5</td>\n",
              "      <td>651.2</td>\n",
              "      <td>1749.0</td>\n",
              "      <td>609.0</td>\n",
              "      <td>4.8</td>\n",
              "      <td>2.3</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>max</th>\n",
              "      <td>41.8</td>\n",
              "      <td>-114.6</td>\n",
              "      <td>52.0</td>\n",
              "      <td>32627.0</td>\n",
              "      <td>6445.0</td>\n",
              "      <td>35682.0</td>\n",
              "      <td>6082.0</td>\n",
              "      <td>15.0</td>\n",
              "      <td>34.2</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "       latitude  longitude  ...  median_income  rooms_per_person\n",
              "count    5000.0     5000.0  ...         5000.0            5000.0\n",
              "mean       35.6     -119.6  ...            3.9               2.0\n",
              "std         2.1        2.0  ...            1.9               1.0\n",
              "min        32.6     -124.3  ...            0.5               0.0\n",
              "25%        33.9     -121.8  ...            2.6               1.5\n",
              "50%        34.3     -118.5  ...            3.6               2.0\n",
              "75%        37.7     -118.0  ...            4.8               2.3\n",
              "max        41.8     -114.6  ...           15.0              34.2\n",
              "\n",
              "[8 rows x 9 columns]"
            ]
          },
          "metadata": {
            "tags": []
          }
        },
        {
          "output_type": "stream",
          "text": [
            "Training targets summary:\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "display_data",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>median_house_value</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>count</th>\n",
              "      <td>12000.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>mean</th>\n",
              "      <td>207.1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>std</th>\n",
              "      <td>115.7</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>min</th>\n",
              "      <td>15.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>25%</th>\n",
              "      <td>120.1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>50%</th>\n",
              "      <td>179.3</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>75%</th>\n",
              "      <td>264.3</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>max</th>\n",
              "      <td>500.0</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "       median_house_value\n",
              "count             12000.0\n",
              "mean                207.1\n",
              "std                 115.7\n",
              "min                  15.0\n",
              "25%                 120.1\n",
              "50%                 179.3\n",
              "75%                 264.3\n",
              "max                 500.0"
            ]
          },
          "metadata": {
            "tags": []
          }
        },
        {
          "output_type": "stream",
          "text": [
            "Validation targets summary:\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "display_data",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>median_house_value</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>count</th>\n",
              "      <td>5000.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>mean</th>\n",
              "      <td>207.7</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>std</th>\n",
              "      <td>116.7</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>min</th>\n",
              "      <td>15.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>25%</th>\n",
              "      <td>118.6</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>50%</th>\n",
              "      <td>182.9</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>75%</th>\n",
              "      <td>266.4</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>max</th>\n",
              "      <td>500.0</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "       median_house_value\n",
              "count              5000.0\n",
              "mean                207.7\n",
              "std                 116.7\n",
              "min                  15.0\n",
              "25%                 118.6\n",
              "50%                 182.9\n",
              "75%                 266.4\n",
              "max                 500.0"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "RWq0xecNKNeG",
        "colab_type": "text"
      },
      "source": [
        " ## 构建神经网络\n",
        "\n",
        "神经网络由 [DNNRegressor](https://www.tensorflow.org/api_docs/python/tf/estimator/DNNRegressor) 类定义。\n",
        "\n",
        "使用 **`hidden_units`** 定义神经网络的结构。`hidden_units` 参数会创建一个整数列表，其中每个整数对应一个隐藏层，表示其中的节点数。以下面的赋值为例：\n",
        "\n",
        "`hidden_units=[3,10]`\n",
        "\n",
        "上述赋值为神经网络指定了两个隐藏层：\n",
        "\n",
        "* 第一个隐藏层包含 3 个节点。\n",
        "* 第二个隐藏层包含 10 个节点。\n",
        "\n",
        "如果我们想要添加更多层，可以向该列表添加更多整数。例如，`hidden_units=[10,20,30,40]` 会创建 4 个分别包含 10、20、30 和 40 个单元的隐藏层。\n",
        "\n",
        "默认情况下，所有隐藏层都会使用 ReLu 激活函数，且是全连接层。"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ni0S6zHcTb04",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def construct_feature_columns(input_features):\n",
        "  \"\"\"Construct the TensorFlow Feature Columns.\n",
        "\n",
        "  Args:\n",
        "    input_features: The names of the numerical input features to use.\n",
        "  Returns:\n",
        "    A set of feature columns\n",
        "  \"\"\" \n",
        "  return set([tf.feature_column.numeric_column(my_feature)\n",
        "              for my_feature in input_features])"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "zvCqgNdzpaFg",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def my_input_fn(features, targets, batch_size=1, shuffle=True, num_epochs=None):\n",
        "    \"\"\"Trains a neural net regression model.\n",
        "  \n",
        "    Args:\n",
        "      features: pandas DataFrame of features\n",
        "      targets: pandas DataFrame of targets\n",
        "      batch_size: Size of batches to be passed to the model\n",
        "      shuffle: True or False. Whether to shuffle the data.\n",
        "      num_epochs: Number of epochs for which data should be repeated. None = repeat indefinitely\n",
        "    Returns:\n",
        "      Tuple of (features, labels) for next data batch\n",
        "    \"\"\"\n",
        "    \n",
        "    # Convert pandas data into a dict of np arrays.\n",
        "    features = {key:np.array(value) for key,value in dict(features).items()}                                             \n",
        " \n",
        "    # Construct a dataset, and configure batching/repeating.\n",
        "    ds = Dataset.from_tensor_slices((features,targets)) # warning: 2GB limit\n",
        "    ds = ds.batch(batch_size).repeat(num_epochs)\n",
        "    \n",
        "    # Shuffle the data, if specified.\n",
        "    if shuffle:\n",
        "      ds = ds.shuffle(10000)\n",
        "    \n",
        "    # Return the next batch of data.\n",
        "    features, labels = ds.make_one_shot_iterator().get_next()\n",
        "    return features, labels"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "U52Ychv9KNeH",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def train_nn_regression_model(\n",
        "    learning_rate,\n",
        "    steps,\n",
        "    batch_size,\n",
        "    hidden_units,\n",
        "    training_examples,\n",
        "    training_targets,\n",
        "    validation_examples,\n",
        "    validation_targets):\n",
        "  \"\"\"Trains a neural network regression model.\n",
        "  \n",
        "  In addition to training, this function also prints training progress information,\n",
        "  as well as a plot of the training and validation loss over time.\n",
        "  \n",
        "  Args:\n",
        "    learning_rate: A `float`, the learning rate.\n",
        "    steps: A non-zero `int`, the total number of training steps. A training step\n",
        "      consists of a forward and backward pass using a single batch.\n",
        "    batch_size: A non-zero `int`, the batch size.\n",
        "    hidden_units: A `list` of int values, specifying the number of neurons in each layer.\n",
        "    training_examples: A `DataFrame` containing one or more columns from\n",
        "      `california_housing_dataframe` to use as input features for training.\n",
        "    training_targets: A `DataFrame` containing exactly one column from\n",
        "      `california_housing_dataframe` to use as target for training.\n",
        "    validation_examples: A `DataFrame` containing one or more columns from\n",
        "      `california_housing_dataframe` to use as input features for validation.\n",
        "    validation_targets: A `DataFrame` containing exactly one column from\n",
        "      `california_housing_dataframe` to use as target for validation.\n",
        "      \n",
        "  Returns:\n",
        "    A `DNNRegressor` object trained on the training data.\n",
        "  \"\"\"\n",
        "\n",
        "  periods = 10\n",
        "  steps_per_period = steps / periods\n",
        "  \n",
        "  # Create a DNNRegressor object.\n",
        "  my_optimizer = tf.train.GradientDescentOptimizer(learning_rate=learning_rate)\n",
        "  my_optimizer = tf.contrib.estimator.clip_gradients_by_norm(my_optimizer, 5.0)\n",
        "  dnn_regressor = tf.estimator.DNNRegressor(\n",
        "      feature_columns=construct_feature_columns(training_examples),\n",
        "      hidden_units=hidden_units,\n",
        "      optimizer=my_optimizer\n",
        "  )\n",
        "  \n",
        "  # Create input functions.\n",
        "  training_input_fn = lambda: my_input_fn(training_examples, \n",
        "                                          training_targets[\"median_house_value\"], \n",
        "                                          batch_size=batch_size)\n",
        "  predict_training_input_fn = lambda: my_input_fn(training_examples, \n",
        "                                                  training_targets[\"median_house_value\"], \n",
        "                                                  num_epochs=1, \n",
        "                                                  shuffle=False)\n",
        "  predict_validation_input_fn = lambda: my_input_fn(validation_examples, \n",
        "                                                    validation_targets[\"median_house_value\"], \n",
        "                                                    num_epochs=1, \n",
        "                                                    shuffle=False)\n",
        "\n",
        "  # Train the model, but do so inside a loop so that we can periodically assess\n",
        "  # loss metrics.\n",
        "  print(\"Training model...\")\n",
        "  print(\"RMSE (on training data):\")\n",
        "  training_rmse = []\n",
        "  validation_rmse = []\n",
        "  for period in range (0, periods):\n",
        "    # Train the model, starting from the prior state.\n",
        "    dnn_regressor.train(\n",
        "        input_fn=training_input_fn,\n",
        "        steps=steps_per_period\n",
        "    )\n",
        "    # Take a break and compute predictions.\n",
        "    training_predictions = dnn_regressor.predict(input_fn=predict_training_input_fn)\n",
        "    training_predictions = np.array([item['predictions'][0] for item in training_predictions])\n",
        "    \n",
        "    validation_predictions = dnn_regressor.predict(input_fn=predict_validation_input_fn)\n",
        "    validation_predictions = np.array([item['predictions'][0] for item in validation_predictions])\n",
        "    \n",
        "    # Compute training and validation loss.\n",
        "    training_root_mean_squared_error = math.sqrt(\n",
        "        metrics.mean_squared_error(training_predictions, training_targets))\n",
        "    validation_root_mean_squared_error = math.sqrt(\n",
        "        metrics.mean_squared_error(validation_predictions, validation_targets))\n",
        "    # Occasionally print the current loss.\n",
        "    print(\"  period %02d : %0.2f\" % (period, training_root_mean_squared_error))\n",
        "    # Add the loss metrics from this period to our list.\n",
        "    training_rmse.append(training_root_mean_squared_error)\n",
        "    validation_rmse.append(validation_root_mean_squared_error)\n",
        "  print(\"Model training finished.\")\n",
        "\n",
        "  # Output a graph of loss metrics over periods.\n",
        "  plt.ylabel(\"RMSE\")\n",
        "  plt.xlabel(\"Periods\")\n",
        "  plt.title(\"Root Mean Squared Error vs. Periods\")\n",
        "  plt.tight_layout()\n",
        "  plt.plot(training_rmse, label=\"training\")\n",
        "  plt.plot(validation_rmse, label=\"validation\")\n",
        "  plt.legend()\n",
        "\n",
        "  print(\"Final RMSE (on training data):   %0.2f\" % training_root_mean_squared_error)\n",
        "  print(\"Final RMSE (on validation data): %0.2f\" % validation_root_mean_squared_error)\n",
        "\n",
        "  return dnn_regressor"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2QhdcCy-Y8QR",
        "colab_type": "text",
        "slideshow": {
          "slide_type": "slide"
        }
      },
      "source": [
        " ## 任务 1：训练神经网络模型\n",
        "\n",
        "**调整超参数，目标是将 RMSE 降到 110 以下。**\n",
        "\n",
        "运行以下代码块来训练神经网络模型。\n",
        "\n",
        "我们已经知道，在使用了很多特征的线性回归练习中，110 左右的 RMSE 已经是相当不错的结果。我们将得到比它更好的结果。\n",
        "\n",
        "在此练习中，您的任务是修改各种学习设置，以提高在验证数据上的准确率。\n",
        "\n",
        "对于神经网络而言，过拟合是一种真正的潜在危险。您可以查看训练数据损失与验证数据损失之间的差值，以帮助判断模型是否有过拟合的趋势。如果差值开始变大，则通常可以肯定存在过拟合。\n",
        "\n",
        "由于存在很多不同的可能设置，强烈建议您记录每次试验，以在开发流程中进行参考。\n",
        "\n",
        "此外，获得效果出色的设置后，尝试多次运行该设置，看看结果的重复程度。由于神经网络权重通常会初始化为较小的随机值，因此每次运行结果应该存在差异。\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "rXmtSW1yKNeK",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "dnn_regressor = train_nn_regression_model(\n",
        "    learning_rate=0.01,\n",
        "    steps=500,\n",
        "    batch_size=10,\n",
        "    hidden_units=[10, 2],\n",
        "    training_examples=training_examples,\n",
        "    training_targets=training_targets,\n",
        "    validation_examples=validation_examples,\n",
        "    validation_targets=validation_targets)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "O2q5RRCKqYaU",
        "colab_type": "text"
      },
      "source": [
        " ### 解决方案\n",
        "\n",
        "点击下方即可查看可能的解决方案"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "j2Yd5VfrqcC3",
        "colab_type": "text"
      },
      "source": [
        " **注意**：在本次练习中，参数的选择有点随意。我们尝试了越来越复杂的组合，并进行了较长时间的训练，直到误差降到目标之下。这决不是最佳组合；其他组合可能会获得更低的 RMSE。如果您的目标是找到可以产生最小误差的模型，那么您需要使用更严格的流程，例如参数搜索。"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "IjkpSqmxqnSM",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 693
        },
        "outputId": "bf713f49-ca88-4bdd-bdce-f847a0edd409"
      },
      "source": [
        "dnn_regressor = train_nn_regression_model(\n",
        "    learning_rate=0.001,\n",
        "    steps=2000,\n",
        "    batch_size=100,\n",
        "    hidden_units=[10, 10],\n",
        "    training_examples=training_examples,\n",
        "    training_targets=training_targets,\n",
        "    validation_examples=validation_examples,\n",
        "    validation_targets=validation_targets)"
      ],
      "execution_count": 7,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "\n",
            "WARNING: The TensorFlow contrib module will not be included in TensorFlow 2.0.\n",
            "For more information, please see:\n",
            "  * https://github.com/tensorflow/community/blob/master/rfcs/20180907-contrib-sunset.md\n",
            "  * https://github.com/tensorflow/addons\n",
            "If you depend on functionality not listed there, please file an issue.\n",
            "\n",
            "Training model...\n",
            "RMSE (on training data):\n",
            "  period 00 : 164.48\n",
            "  period 01 : 163.13\n",
            "  period 02 : 142.31\n",
            "  period 03 : 133.48\n",
            "  period 04 : 126.33\n",
            "  period 05 : 117.48\n",
            "  period 06 : 110.74\n",
            "  period 07 : 107.68\n",
            "  period 08 : 106.54\n",
            "  period 09 : 105.04\n",
            "Model training finished.\n",
            "Final RMSE (on training data):   105.04\n",
            "Final RMSE (on validation data): 106.10\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAacAAAEYCAYAAAD4czk4AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzs3Xd4FFX3wPHvSS+kkUIJLfQuJTTp\nRQVEUCxge+0oIuBrLz97fe0FG1gQERCxgAioSG9CqNJ7SQIkAQKE9OT+/pgBNgUSIMmmnM/z7LO7\nd+7Mnp3d5Oy9c+eOGGNQSimlShMXZweglFJK5abJSSmlVKmjyUkppVSpo8lJKaVUqaPJSSmlVKmj\nyUkppVSpo8lJqTJCRIyI1Hd2HGWZiDwjIl9e5LrjReTVoo5J5U+TUzknIntFJEVEkkTkkP0HVqkI\ntlvH/mfpdp46L9p1RucqH22Xv3ipcVwoEekiIstE5LiIHBWRpSLSrqTjKGoiskBEUu3P+fTtN2fH\nVRTs78op+z3FiMh7IuJ6MdsyxrxujLm3qGNURU+TU8VwjTGmEtAKaA08XYKvvR34T66yO+zyEiUi\n/sBM4GOgMhAOvASkOSGWi/rnWoCHjDGVHG7XnOO18/ygON+PjMJuo5hdZn+HewO3APdd6AacELO6\nBJqcKhBjzCHgD6wkBYCIBIjIBBGJF5F9IvJ/IuJiL3Oxn+8TkTi7XoC96iL7PtH+RdvpHC+7CvAR\nkWb2NpsBXnb5GSIyQETWiUii3bJp6bDsKRHZJSInRWSziFznsOxOEVkiIu+IyDER2SMi/c4RS0N7\nP0w2xmQZY1KMMX8aYzbY23K1t5MgIrtFZIRj69BuhfZxeO0XRWSiw/Mf7dbpcRFZdPo928vGi8hn\nIjJLRE4BPUXE0369/SJyWEQ+FxFvh3UeF5GDIhIrInef4z0VSER6iEi0iDwpIoeAb/Irs+veJyI7\n7VblDBGp7rAdY++THcCOfF5ntog8lKtsvYgMFsv79vfohIj8KyLNL/S9GGO2AouB5vb2q4vIT/b3\nd4+IjHJ47RdFZJqITBSRE8Cd+XxmA0Vkk/29WyAiTRyWtRaRNfb37ges7+3pZSEiMtNe76iILD79\nd6OKhu7MCkREagD9gJ0OxR8DAUBdoDtWK+cue9md9q2nvbwSMMZe1s2+D7R/pS8/z0t/x9nW0x32\nc8e4WgNfA/cDwcAXwAwR8bSr7AK62nG+BEwUkWoOm+gAbANCgLeAr0RE8oljO5AlIt+KSD8RCcq1\n/D5gAFbrMhK44TzvKT+zgQZAGLAG+D7X8luA1wA/YAnwJlbCbAXUx2rJPQ8gIn2Bx4Ar7G324dJU\nxWot1gaG5VcmIr2AN4CbgGrAPmBKru1ci7W/m+bzGpOBm08/EZGm9rZ/B67E+s40xPocbwKOXOib\nsLfZFVhrJ4PfgPVY+6438LCIXOWwyiBgGhBIrs9DRBraMT8MhAKzgN9ExENEPIBfsb6rlYEfgesd\nVn8UiLbXqwI8A+hccEXJGKO3cnwD9gJJwEmsP56/sRIKgCuQDjR1qH8/sMB+/DfwoMOyRkAG4AbU\nsbfndp7XfhGYCNQC9gPu9n1Nu/xFu95nwCu51t0GdD/HdtcBg+zHdwI7HZb52HFVPce6TYDxWP9Y\nMoEZQBV72TzgAYe6Vzq+R3tf9sn9/s7xOoH2ugH28/HABIflApwC6jmUdQL22I+/Bt50WNbQ3l79\nc7zeAiAZSHS4vWIv62F/zl4O9fMr+wp4y+F5JfvzrmM/N0Cv83zefvZ7qm0/fw342n7cC+vHQUfA\n5QK/wwY4ARzD+qHyKtYP6w7A/lx1nwa+cfh8FuX3nbQfPwdMdVjmAsTY+6YbEAuIw/JlwKv245eB\n6ef6PPR26TdtOVUM1xpj/LD+6BpjtTCw792xfiGftg/rVyhA9XyWuWH9Uiw0Y8x+rNba68AOY8yB\nXFVqA4/aXSSJIpKIlcCqA4jIfxy6/BKxunRCHNY/5PBayfbDfAd9GGO2GGPuNMbUsLdTHfjA4f06\nxrYv9/rnYncJvml3P57ASmTkitNx26FYiXS1w/uaY5dfbCyjjDGBDrfnHJbFG2NSc9XPXZbj8zbG\nJGG1bsId6uT+7M4wxpzEaiUNtYtuxm6tGGPmYbW6PwHiRGSsWMcAC6uNMSbIGFPPGPN/xphsrO9N\n9Vzfm2fI+f08Z7zkfb/Zdv1we1mMsTORzfEzeBvrO/2n3QX81AW8F1UImpwqEGPMQqxf8O/YRQlY\nv4xrO1SrhfXrEaxfjrmXZQKHufAujAlYXSET8ll2AHgt1z9WH2PMZBGpDYwDHgKCjTGBwEaslscl\nMdbxi/HYxy+Ag1hJ8bRauVY5hZVQTqvq8PgWrC6kPljdVnXscsc4HfdZApACNHN4zwHGOuhfmFgu\nVH6fV+6yHJ+3iPhidbPGnGed3CYDN4t1DNILmH9mRWM+Msa0xeoSbAg8Xujo83cAq6Xp+L3xM8b0\nL2S8ud+vYO3zGKz9H56re/jMZ2CMOWmMedQYUxcYCDwiIr0v8f0oB5qcKp4PgCtE5DJjTBYwFXhN\nRPzsRPAIVpcbWP9o/isiEWINP38d+MEYkwnEA9lYx6IK4wesbrKp+SwbBzwgIh3sA+e+InK1iPgB\nvlj/YOIBROQuziaTCyIijUXkUfvYGyJSE+vX/Qq7ylRglIjUsI9H5f41vA4YKiLuIpL7mJQf1qi/\nI1gJ7PXzxWL/Sh8HvC8iYXY84Q7HS6ZiHcBvKiI+wAsX854v0GTgLhFpZR/vex34xxiz9wK2MQvr\nH/7LWN+VbAARaWd/vu5YST4V6/tzKVYCJ8Ua1OFtt16bS+FPDZgKXC0ive24HsX6DJcBy7F+iI2y\nP+/BQPvTK4o1gKe+nbyOA1lF8H6UA01OFYwxJh6r9fK8XTQS65/FbqyD9JOwjndg33+HNTJvD9Y/\nlJH2dpKxjikstbtUOhbwuinGmLnGmJR8lkVhDUYYg3VcYSfWsSSMMZuBd7H+WRwGWgBLL+Ktg3Xc\nrQPwj1gj5lZgtcIetZePwxrNuB5rQMPPudZ/Dqhnx/gS1r46bQJWt08MsJmzCe98nsR6ryvsrsC5\nWMf1MMbMxvohMc+uM68Q2xsjOc9zWl2Idc4wxszFeo8/YbUc6nG2i66w20jD2m99yLl//LH27zGs\n/XQEq2vs9Imxsy/kdezXysIawNIK6/uZAHyJ1XItzPrbgNuwBgUlANdgnXaRboxJBwZjfQ+PAkPI\n+X1ogPV5JWF9Nz81xsxHFRnJ2aWqlDpNROpg/dNzt1uLSqkSoi0npZRSpU6xJScR+VqsE+42OpT9\nYI+6WifWCY3rHJY9LdbJf9tynaeglFKqgim2bj0R6YbVHzvBGJPnALaIvAscN8a8bJ9YNxnrgGN1\nrL7chnafslJKqQqm2FpOxphFWAcS87BHuNyElZDAGoI7xRiTZozZg3UAuH1+6yqllCr/nDURYlfg\nsDHm9Pxc4eQc3RRNzhP/zhCRYdjTr/j6+rZt3LhxccaplFKqCK1evTrBGBNaUD1nJaebOdtquiDG\nmLHAWIDIyEgTFRVVlHEppZQqRiJSqJlXSjw5iTXD82CgrUNxDDnPhq9BzrPSlVJKVSDOGEreB9hq\njIl2KJuBdea9p4hEYJ3gttIJsSmllCoFinMo+WSsM6cbiXXdmHvsRUPJ1aVnjNmENZXIZqzJL0fo\nSD2llKq4yvQMEXrMSSlVFDIyMoiOjiY1NffE7epieXl5UaNGDdzd3XOUi8hqY0xkQevrZYuVUhVe\ndHQ0fn5+1KlTh/yvU6kuhDGGI0eOEB0dTURExEVtQ6cvUkpVeKmpqQQHB2tiKiIiQnBw8CW1RDU5\nKaUUaGIqYpe6Pyt2t976H8DDBypVOXtz93J2VEopVeFV7OT0+yOQnpSzzDMA/KrkTFh5nlcF7yDQ\nX1pKqSKSmJjIpEmTePDBBy9ovf79+zNp0iQCAwPPWef555+nW7du9OnT51LDLDEVe7TeycOQ5HA7\neQiS4iDJvj95yCrPSM67rou7nazCrGRVKQwqnb6v4lBWBdw8Lz5GpVSx27JlC02aNHFqDHv37mXA\ngAFs3LgxR3lmZiZubmWzHZHfftXReoXhZ7eKCpJ2MmeyOpPM7PvE/RC9Ck4lYF1RPBevwLwJzK9q\n3taZV6C2xpSqoJ566il27dpFq1atcHd3x8vLi6CgILZu3cr27du59tprOXDgAKmpqYwePZphw4YB\nUKdOHaKiokhKSqJfv3506dKFZcuWER4ezvTp0/H29ubOO+9kwIAB3HDDDdSpU4c77riD3377jYyM\nDH788UcaN25MfHw8t9xyC7GxsXTq1Im//vqL1atXExIS4pT9UbGTU2F5+lm34Hrnr5eVCafi809g\np1tjB/6xnmfmM4rF1dNKWv3fhoZ6SSulnOGl3zaxOfZEkW6zaXV/Xrim2XnrvPnmm2zcuJF169ax\nYMECrr76ajZu3HhmKPbXX39N5cqVSUlJoV27dlx//fUEBwfn2MaOHTuYPHky48aN46abbuKnn37i\ntttuy/NaISEhrFmzhk8//ZR33nmHL7/8kpdeeolevXrx9NNPM2fOHL766qui2wEXQZNTUXJ1A/9q\n1u18jIG0E7laY3Z34paZ8PujULeHdgcqVYG1b98+xzlCH330Eb/88gsABw4cYMeOHXmSU0REBK1a\ntQKgbdu27N27N99tDx48+Eydn3/+GYAlS5ac2X7fvn0JCgoq0vdzoTQ5OYMIeAVYt5AGOZfV7Qnf\nXQurx0OH+50SnlIVWUEtnJLi6+t75vGCBQuYO3cuy5cvx8fHhx49euR7DpGn59kftK6urqSkpOS7\n7dP1XF1dyczMLOLIi4ae51Ta1O0BdbrCorch/ZSzo1FKlRA/Pz9OnjyZ77Ljx48TFBSEj48PW7du\nZcWKFfnWuxSdO3dm6tSpAPz5558cO3asyF/jQlTo5PTpgp18u2wvf20+zKbY4xxPzsDpoxdFoNdz\n1rGrf75wbixKqRITHBxM586dad68OY8//niOZX379iUzM5MmTZrw1FNP0bFjxyJ//RdeeIE///yT\n5s2b8+OPP1K1alX8/PyK/HUKq8IOJTfGcNlLf3IiNWeT1tfDleqB3lQP9CY8yJvwQG+qB3pRPcB6\nXsXfC3fXEsjpk4bA/uUwegN4n/v8BaXUpSsNQ8mdLS0tDVdXV9zc3Fi+fDnDhw9n3bp1l7RNHUp+\nEUSE9S9cSUJSOrGJKcQmphCTmEJsYioxicnEJqayMeY4R06l51jPRaCKv9fZBBboTXigV46E5u/l\nfo5XvQC9/g8+7wLLPobez1369pRS6jz279/PTTfdRHZ2Nh4eHowbN86p8VTY5ARWggr18yTUz5PL\naubfOklJz+Lg8dOJK4WYxFTr/lgKG6IT+WPjIdKzsnOs4+fpZicrL8KDziax0/dhfp64FdT6qtoC\nmg2GFZ9BhwegUmhRvW2llMqjQYMGrF271tlhnFGhk1NheHu4Uje0EnVDK+W7PDvbkHAqjZhjVqvr\nbAvMul93IJFjyRk51nF1Ear6e1ndhbkSV40gb+qHVbImTez5LGyeDkveg75vlMTbVUqpUkGT0yVy\ncRHC/LwI8/Oida386ySnZ9rdhSm5uhBTWLP/GL9vOEhm9tljf6N61eeRKxtBSH1odQus+hI6PgiB\nNUvoXSmllHNpcioBPh5u1A+rRP2w/FtfWdmGhKQ0YhJT+GLhLj5ftJub2tWkRpAPdH8SNvwAi96C\ngR+XcORKKeUcFXooeWnh6iJU8feiTa0gXrimGQK8++d2a2FgTYi8G9Z+D0d2OTVOpZQqKZqcSpnq\ngd7c3SWCX9bGsDHmuFXY9VFrKqP5rzs3OKVUqVGpktUTExsbyw033JBvnR49elDQ6TYffPABycln\nr7zQv39/EhMTiy7Qi6TJqRQa3qMelX09eH3WFuuk4Eph0HE4bJwGh/51dnhKqVKkevXqTJs27aLX\nz52cZs2add5rQ5UUTU6lkL+XO6N61WfZriMs2BZvFV4+0pqLb95rzg1OKVUsnnrqKT755JMzz198\n8UVeffVVevfuTZs2bWjRogXTp0/Ps97evXtp3rw5ACkpKQwdOpQmTZpw3XXX5Zhbb/jw4URGRtKs\nWTNeeOEFwJpMNjY2lp49e9KzZ0/AugRHQkICAO+99x7NmzenefPmfPDBB2der0mTJtx33300a9aM\nK6+88pxz+F0KHRBRSt3SoTbjl+3ljdlb6NogBDfvILh8FMx7BQ6shJrtnR2iUuXT7KeKvoeiagvo\n9+Z5qwwZMoSHH36YESNGADB16lT++OMPRo0ahb+/PwkJCXTs2JGBAwdap5rk47PPPsPHx4ctW7aw\nYcMG2rRpc2bZa6+9RuXKlcnKyqJ3795s2LCBUaNG8d577zF//vw8121avXo133zzDf/88w/GGDp0\n6ED37t0JCgoq9KU5LoW2nEopDzcXnuzbmO2Hk5i2Otoq7PAA+IbC3y87NzilVJFr3bo1cXFxxMbG\nsn79eoKCgqhatSrPPPMMLVu2pE+fPsTExHD48OFzbmPRokVnkkTLli1p2bLlmWVTp06lTZs2tG7d\nmk2bNrF58+bzxrNkyRKuu+46fH19qVSpEoMHD2bx4sVA4S/NcSmKreUkIl8DA4A4Y0xzh/KRwAgg\nC/jdGPOEXf40cI9dPsoY80dxxVZW9G1elba1g3jvr+0MbFUdH89K0PUxmPMk7F5gzWCulCpaBbRw\nitONN97ItGnTOHToEEOGDOH7778nPj6e1atX4+7uTp06dfK9VEZB9uzZwzvvvMOqVasICgrizjvv\nvKjtnFbYS3NciuJsOY0H+joWiEhPYBBwmTGmGfCOXd4UGAo0s9f5VERcizG2MkFEeKZ/Y+JOpjFu\n0R6rMPIu8K9htZ7K8KS9Sqm8hgwZwpQpU5g2bRo33ngjx48fJywsDHd3d+bPn8++ffvOu363bt2Y\nNGkSABs3bmTDhg0AnDhxAl9fXwICAjh8+DCzZ88+s865LtXRtWtXfv31V5KTkzl16hS//PILXbt2\nLcJ3e37FlpyMMYuAo7mKhwNvGmPS7DpxdvkgYIoxJs0YswfYCehBFaBt7cr0bVaVLxbtIv5kmjWk\nvMeTELMats1ydnhKqSLUrFkzTp48SXh4ONWqVePWW28lKiqKFi1aMGHCBBo3bnze9YcPH05SUhJN\nmjTh+eefp23btgBcdtlltG7dmsaNG3PLLbfQuXPnM+sMGzaMvn37nhkQcVqbNm248847ad++PR06\ndODee++ldevWRf+mz6FYL5khInWAmae79URkHTAdq3WUCjxmjFklImOAFcaYiXa9r4DZxpg84yNF\nZBgwDKBWrVptC/olUR7sSTjFFe8tZEi7mrx2XQvIyoRP2luJ6oEl4FLhG5lKXRK9ZEbxuJRLZpT0\ngAg3oDLQEXgcmCrnGnZyDsaYscaYSGNMZGhoxZipOyLEl1s71GLKqgPsjEsCVzfo9SzEbYaNPzs7\nPKWUKnIlnZyigZ+NZSWQDYQAMYDjrKY17DJlG9W7AT7urrw5e6tV0PQ6qNIC5r8GWRnnX1kppcqY\nkk5OvwI9AUSkIeABJAAzgKEi4ikiEUADYGUJx1aqBVfy5IEe9Zi75TD/7D4CLi7WBQmP7YG1E50d\nnlJlXlm+KnhpdKn7s9iSk4hMBpYDjUQkWkTuAb4G6orIRmAKcIfditoETAU2A3OAEcaYrOKKray6\np0sE1QK8eH3WFrKzDTS8Cmq0h4VvQcbFDwtVqqLz8vLiyJEjmqCKiDGGI0eO4OXlddHbKNYBEcUt\nMjLSFDSpYXkzbXU0j/24no9ubs3Ay6rDnsXw7QC46nXoNMLZ4SlVJmVkZBAdHX1J5/6onLy8vKhR\nowbu7u45ygs7IEKnLypjrmsdzldL9vDWnK1c1awKnhFdoW5PWPwutPkPePo5O0Slyhx3d3ciIiKc\nHYZyoNMXlTGuLtaJudHHUvhuuT2MvvdzkHwEVnzm3OCUUqqIaHIqg7o2CKVrgxA+nreT48kZEN4W\nGg+AZR9Dcu7znpVSquzR5FRGPdO/CSdSM/hkwU6roOezkHYSln7o3MCUUqoIaHIqo5pU8+f6NjUY\nv3QvB44mQ5Wm0PIm+OcLOHnI2eEppdQl0eRUhj16ZUNcXOCdP7dZBT2eguwMWPSOcwNTSqlLpMmp\nDKsW4M09XSKYvi6WDdGJULkutL4dVo+HY3udHZ5SSl00TU5l3APd6xHs68Hrs7ZYJxB2fwLEBRb8\nz9mhKaXURdPkVMb5ebkzuk8DVuw+yrytceBfHdrfBxumQPw2Z4enlFIXRZNTOXBz+1rUDfHljdlb\nyczKhi6PgLuPNSmsUkqVQZqcygF3Vxee6NuYnXFJTI2KBt9gayqjzdMhdq2zw1NKqQumyamcuKpZ\nFSJrB/H+3O2cSsu0kpN3EMx71dmhKaXUBdPkVE6ICE/3b0L8yTTGLd4NXgHQ5b+wcy7sW+bs8JRS\n6oJocipH2tYOon+LqoxdtJu4k6nQ7j6oVBX+fhnK8OzzSqmKR5NTOfPEVY3JyMrm/b92gIcPdHsM\n9i+HnX87OzSllCo0TU7lTJ0QX27tUJsfVu1nx+GT0OYOCKwF87T1pJQqOzQ5lUOjejfA18ONN2dv\nBTcP6PEMHFwPW2Y4OzSllCoUTU7lUGVfDx7sWZ+/t8axfNcRa0LYkEbWyL3sLGeHp5RSBdLkVE7d\n1bkO1QO8eH3WFrJxgV7PQsJ22PCDs0NTSqkCaXIqp7zcXXnsqkb8G3Oc3zbEQpOBUK0VLHgDMtOd\nHZ5SSp2XJqdy7NpW4TSt5s9bc7aRlpVtXc49cT+s+dbZoSml1HlpcirHXFyEZ/o3ISYxhQnL9kG9\n3lC7Myx6G9KTnR2eUkqdkyancq5LgxC6Nwzl43k7SEzJgF7PQdJhWDnW2aEppdQ5FVtyEpGvRSRO\nRDY6lL0oIjEiss6+9XdY9rSI7BSRbSJyVXHFVRE93b8xSWmZjJm3E2p3gvpXwJL3IfW4s0NTSql8\nFWfLaTzQN5/y940xrezbLAARaQoMBZrZ63wqIq7FGFuF0riqPze0rcGE5fs4cDQZev0fpCbC8k+c\nHZpSSuWr2JKTMWYRcLSQ1QcBU4wxacaYPcBOoH1xxVYRPXJFI1xc4K0/tkH1VtD0Wis5nUpwdmhK\nKZWHM445PSQiG+xuvyC7LBw44FAn2i7LQ0SGiUiUiETFx8cXd6zlRtUAL+7rWpff1sey/kAi9HwW\nMpKt7j2llCplSjo5fQbUA1oBB4F3L3QDxpixxphIY0xkaGhoUcdXrt3fvR4hlTx4bdYWTEgDuOxm\nWDkOjsc4OzSllMqhRJOTMeawMSbLGJMNjONs110MUNOhag27TBWhSp5ujO7TkJV7jjJ3Sxx0fxJM\nNix6y9mhKaVUDiWanESkmsPT64DTI/lmAENFxFNEIoAGwMqSjK2iGNquJnVDfHlz9hYy/WtC5F2w\ndiIc2eXs0JRS6oziHEo+GVgONBKRaBG5B3hLRP4VkQ1AT+C/AMaYTcBUYDMwBxhhjNEZSouBu6sL\nT/ZrzK74U/wQdQC6PgYu7rDgTWeHppRSZ4gpw9f4iYyMNFFRUc4Oo8wxxnDTF8vZk5DMgsd7UGnR\nK7D0Qxi+FKo0c3Z4SqlyTERWG2MiC6qnM0RUQCLWtEYJSWmMXbQbOo8GTz+Y95qzQ1NKKUCTU4XV\nulYQV7esxrhFuzmc6QOXj4Jtv0P0ameHppRSmpwqsievakxmdjbv/7UdOj4APiHW5dyVUsrJNDlV\nYLWCfbi9Yx2mRh1geyLQ9RHYvQB2L3R2aEqpCk6TUwU3sld9fD3deGPWFoi8B/zDYd4rUIYHyiil\nyj5NThVckK8HD/Wsz/xt8SzblwTdn4DoVbD9D2eHppSqwDQ5Ke64vA7hgd68PnsL2S1vgcp1rdZT\ndrazQ1NKVVCanBRe7q48dlVDNsacYMbGeOjxDBzeCJt+dnZoSqkKSpOTAmDQZeE0D/fn7T+2kdr4\nWghrBvNfh6xMZ4emlKqANDkpAFxchGf6NSEmMYVvl++3Lkh4dBesn+Ts0JRSFZAmJ3XG5fVD6Nko\nlDHzd3KsRm8Ij4QF/4OMVGeHppSqYDQ5qRye7t+EU2mZfDx/F/R+Dk5Ew+pvnB2WUqqC0eSkcmhY\nxY+bImvy3Yq97AtoBxHdYNE7kJbk7NCUUhWIJieVxyNXNMTNxYW3/tgGvZ6H5AT44VY4dcTZoSml\nKghNTiqPMH8v7utWl983HGStqQ8Dx8C+5fBFVziwytnhKaUqAE1OKl/DutUlpJIHb8zaiml9G9zz\nJ7i4wjf9YOU4nd5IKVWsNDmpfFXydOPhPg1Zufcof20+DNVbwf2LoH5vmPUY/HSvHodSShUbTU7q\nnIa2q0m9UF/enLOVjKxs8A6CoZOh13PW7BFf9ob47c4OUylVDmlyUufk5urCU/2asDv+FM/+8i+p\nGVng4gLdHoPbf4FTCTCuJ2zUaY6UUkVLk5M6rz5NwhjRsx5To6K58fPlHDiabC2o28Pq5gtrCtPu\ngjlPQ1aGM0NVSpUjmpzUeYkIj1/VmHH/iWTvkVMM+HgJ87YethYGhMOdv0OH4bDiUxh/NZyIdW7A\nSqlyQZOTKpQrmlZh5sguhAd6c/f4KN75YxtZ2QbcPKDfm3DDN3BoI3zeVa+kq5S6ZJqcVKHVDvbl\n5wcvZ0hkTcbM38l/vv6HI0lp1sLmg2HYfPAJhu+uhcXv6vWglFIXrdiSk4h8LSJxIrIxn2WPiogR\nkRD7uYjIRyKyU0Q2iEib4opLXRovd1f+d0NL3rq+JVF7j3H1R0tYve+YtTC0Edw3D5oNhr9fhim3\nQMox5waslCqTirPlNB7om7tQRGoCVwL7HYr7AQ3s2zDgs2KMSxWBm9rV5Kfhl+Ph5sKQL5bzzdI9\nGGPAsxJc/yX0ext2zoUvusPB9c4OVylVxhRbcjLGLAKO5rPofeAJwHGKgUHABGNZAQSKSLXiik0V\njebhAfw2sgs9GoXx0m+beWjyWpLSMkEEOgyDu2ZDdiZ8eQWsmeDscJVSZUiJHnMSkUFAjDEm90/p\ncOCAw/Nouyy/bQwTkSgRiYr3nmm2AAAgAElEQVSPjy+mSFVhBXi7M/b2tjzZtzGz/z3IwDFL2H74\npLWwZjtruHntTjBjJEwfARkpzg1YKVUmlFhyEhEf4Bng+UvZjjFmrDEm0hgTGRoaWjTBqUvi4iIM\n71GPifd24ERKBoPGLGX6uhhroW8I3PYzdHsC1k6Er66Ao3ucG7BSqtQ7b3ISkV4OjyNyLRt8ga9V\nD4gA1ovIXqAGsEZEqgIxQE2HujXsMlWGXF4vhN9HdaV5uD+jp6zj+ekbScvMsiaM7fUs3DIVEg9Y\nx6G2znJ2uEqpUqygltM7Do9/yrXs/y7khYwx/xpjwowxdYwxdbC67toYYw4BM4D/2KP2OgLHjTEH\nL2T7qnSo4u/FpPs6cl/XCCYs38dNX6wgJtHuymt4ldXNVzkCptwMc1+CrEznBqyUKpUKSk5yjsf5\nPc+5UGQysBxoJCLRInLPearPAnYDO4FxwIMFxKVKMXdXF569uimf3dqGXXFJDPhoMYu228cHg2rD\n3X9A2zthyXvWOVFJcU6NVylV+og5z3V5RGSNMaZN7sf5PXeGyMhIExUV5cwQVAF2xycxfOIatsed\nZHTvBozq1QAXF/t3zbpJMPO/1mznN46HWh2dGqtSqviJyGpjTGRB9QpqOdUVkRki8pvD49PPIwpY\nVynqhlbilxGXc12rcD6Yu4M7x6/i6Kl0a2GrW+Dev8Hd25qXb/mnehFDpRRQcMup+/lWNsY4dRI1\nbTmVHcYYJq3cz0szNhPq58knt7ahVc1Aa2Hqcfj1Qdg6E5peC4PGgKefcwNWShWLwraczpuc8tmo\nO9Ac61wlpx8o0ORU9myITmT4xDXEnUzl+QFNua1jbUTEajEt+8gaJFG5Lgz5DsKaODtcpVQRK5Ju\nPRH5XESa2Y8DgPXABGCtiNxcJJGqCqVljUB+H9WFzvVDeG76Jv77wzqS0+1ZJTqPhjtmWC2pcb1g\nw4/ODlcp5SQFHXPqaozZZD++C9hujGkBtMWagkipCxbo48HXd7Tj0SsaMn19LNd+spRd8UnWwjpd\n4IHFUK0V/Hwv/P4YZKY5N2ClVIkrKDmlOzy+AvgVwD43SamL5uIijOzdgAl3tychKZ2BHy/h9w32\nqW1+Va0W1OUjYdU4+KafdfKuUqrCKCg5JYrIABFpDXQG5gCIiBvgXdzBqfKva4NQZo7sQsOqfoyY\ntIaXf9tMRlY2uLrDla/CTd9B/Hb4ohvs/NvZ4SqlSkhByel+4CHgG+BhhxZTb+D34gxMVRzVA735\nYVgn7ry8Dl8v3cPQsSs4dDzVWth0IAxbYLWmJl4PC/6nFzFUqgK4oNF6pY2O1it/flsfy5M/bcDb\n3ZWPbm5N5/oh1oL0UzDzEdgwBepfAYPHgk9l5warlLpgRTKUXEQ+Ot/KxphRFxFbkdHkVD7tjDvJ\nAxPXsDs+iUevbMTw7vWsWSWMgdXfwOwnoVIVuPpdaHClNdJPKVUmFNUMEQ8AXYBYIApYneumVJGr\nH+bH9BGdGdCyOm//sY37JkRxPDnDSkKRd1tz87m6w6Sb4Jv+cGCls0NWShWxglpOwcCNwBAgE/gB\nmGaMSSyZ8M5PW07lmzGGCcv38ervm6ka4MVnt7aleXiAtTAzHdZ8CwvfglNx0HgA9H4eQhs5N2il\n1HkVScvJGHPEGPO5MaYn1nlOgcBmEbm9iOJU6pxEhDsur8MP93ciK8sw+LNlTF65H2MMuHlA+/tg\n1Fro+X+weyF82tG62u7xaGeHrpS6RIW6Eq6ItAFGA7cBs9EuPVWC2tQKYuaornSIqMzTP//LYz9u\nICU9y1roWQm6Pw6j10GHB2DDVPioDfz5f5B81LmBK6UuWkHdei8DVwNbgCnAHGNMqbk6nHbrVSxZ\n2YYP/97Bx/N20KiKHx8ObU2jqrkmiD22Dxa8AeungKc/dHnYSloePs4JWimVQ1GN1ssG9gDJdtHp\nygIYY0zLSw30UmhyqpgWbIvjkanrOZGSwX3d6jKqVwO8PVxzVjq8Cf5+GbbPAb9q0P1JaH07uLo5\nJ2ilFFB0yan2+VY2xuy7iNiKjCaniuvoqXTemLWFH1dHU7OyN68Mak6PRmF5K+5bBn+9ANErIbi+\nNWiiyUAdfq6UkxTLJTMcNu4C3GyM+f5igisqmpzU8l1HePbXf9kdf4prLqvOcwOaEObnlbOSMbBt\nltWSit8K4W2hz4sQ0c0ZIStVoRXVJTP8ReRpERkjIleKZSSwG7ipqIJV6mJ1qhfM7NFd+W+fhvyx\n8RC9313IxBX7yM52+NElAo2vhuHLYNAncPIwfHsNfDcYDq53XvBKqXMqqFtvOnAMWI41n14Y1vGm\n0caYdSUS4Xloy0k52h2fxP/9upFlu47QplYgrw9uQeOq/nkrZqRas50vfhdSjkHzG6DXs9ZFDpVS\nxaqojjn9a1+/CRFxBQ4CtYwxqUUW6SXQ5KRyM8bwy9oYXv19CydSMri3a11G985nwARASqJ19d3l\nn0J2BrS9C7o/AZXyOXallCoSRTV9UcbpB8aYLCC6tCQmpfIjIgxuU4O/H+nO4DbhfL5wF1e8v5D5\n2+LyVvYOtAZIjFprjeSL+ho+bAXzX4fUEyUfvFLqjIJaTlnAqdNPsa7hlMzZoeT59JmUHG05qYKs\n2H2EZ3/5l13xp7i6ZTVeGNCUMH+v/Csn7IR5r8DmX8EnGLo9bs3l5+ZZskErVY4V62i9QgbwNTAA\niDPGNLfLXgEGAdlAHHCnMSZWRAT4EOiPlfzuNMasKeg1NDmpwkjLzGLswt18PH8nnq4uPNGvMbe2\nr2XNdJ6fmNUw90XYswgCa0HPZ6HFjeCST9egUuqCFFW33qUYD/TNVfa2MaalMaYVMBN43i7vBzSw\nb8OAz4oxLlXBeLq5MrJ3A/54uBstawbw3K8buf7zZWw5eI6uu/C28J8ZcNvP4BUIv9xvXYl3+5/W\nsHSlVLErtuRkjFkEHM1V5vjfwJezM04MAiYYywogUESqFVdsqmKKCPFl4j0d+GBIK/YfSWbAx0t4\nY9YWktPzmZFLBOr3hmEL4fqvrIsdTroRxl8NB1aVfPBKVTDF2XLKl4i8JiIHgFs523IKBw44VIu2\ny/Jbf5iIRIlIVHx8fPEGq8odEeHa1uH8/Wh3bmxbgy8W7eaK9xYxf2s+AyYAXFygxQ0wYiX0fwcS\ndsBXfWDKrRC/rWSDV6oCKfHkZIx51hhTE/geeOgi1h9rjIk0xkSGhoYWfYCqQgj08eDN61sy9f5O\neHu4ctf4VYz4fg2HT5xjMKpeokOpElXiycnB98D19uMYoKbDshp2mVLFqn1EZWaN6spjVzbkry2H\n6fPuQiYs30tW9jmOLZ25RMf6XJfoeE4v0aFUESrR5CQiDRyeDgK22o9nAP+xp0fqCBw3xhwsydhU\nxeXh5sJDvRrw58PdaFUrkOenb2LwZ8vYFHv83Cv5BkPfN+ChKGg+GJZ9DB+1gs0zSi5wpcqx4hxK\nPhnoAYQAh4EXsIaKN8IaSr4PeMAYE2MPJR+DNbovGbjLGFPgGHEdSq6KmjGGGetjeWXmZo4lZ3BP\nlwge7tMAH48CLrVxeBPMGAUxUdb5UT2esY5XKaVycPp5TiVBk5MqLonJ6fxvzlYmrzxAeKA3Lw9q\nRu8mVc6/UmYa/P4IrJ0IDfvC4LHgFVAyAStVRpSG85yUKrMCfTx4Y3BLfnygEz4ertzzbRTDJ67m\n0PHzzN7l5gkDx1ij+nbOhXG9rdF9SqkLpslJqfNoV6cyv4/qyuNXNWLe1jj6vLeQb5edZ8CEiDWq\n7z/TrRnPx/WCbXNKNmilygFNTkoVwMPNhRE96/Pnf7vRulYgL8zYxOBPl55/wESdLjBsAVSOgMlD\nYdHbOruEUhdAk5NShVQ72JcJd7fnw6GtiElMYeCYpbw6czOn0vKZYQIgsCbcNceal2/eqzD1P5CW\nVLJBK1VGaXJS6gKICINahfP3Iz24KbImXy7ZwxXvLWTu5sP5r+DhYw2MuPI12DoTvroCju4u2aCV\nKoM0OSl1EQJ83HljcAumPdAJPy937p0Qxf3fRZGQlJa3sghc/pA1keyJWBjbE3bNK/mglSpDNDkp\ndQki61Rm5qguPNm3MfO3xTPgoyWs3ncs/8r1elrHofzDYeL1sPQjPQ6l1DloclLqErm7ujC8Rz1+\nHn45Hm4uDPliOd8s3UO+5xBWjoB7/oQm18Bfz8HP90F6cskHrVQpp8lJqSLSPDyA30Z2oUejUF76\nbTMjJ6/Nf7CEZyW48Vvo9Rz8Ow2+vgoS95d8wEqVYpqclCpCAd7ujL09kif6NmLWvwcZ9MlSdsad\nzFtRBLo9Brf8AMf2wtgesHdJSYerVKmlyUmpIubiIjzYoz4T7+1AYnI6A8csZcb62PwrN7wK7psH\nPsEwYRD8M1aPQymFJielis3l9UKYObIrTar5M2ryWl6csYn0zOy8FUMawL1/Q/0rYPbjMOMhyDjP\nNElKVQCanJQqRlUDvJgyrCN3d45g/LK9DB27nIPHU/JW9PKHoZOg+5PWxLHjr4YTetUYVXFpclKq\nmLm7uvD8NU0Zc0trth06ydUfLWHpzoS8FV1coOczcNN3ELcFxnaHAytLPmClSgFNTkqVkAEtqzP9\noS4E+3pw+1f/MGbeDrLzm0C26UC4dy64+8A3/WH1tyUfrFJOpslJqRJUP6wSv47ozICW1Xnnz+3c\nOyGK48kZeStWaQrD5kNEN/htFMx8BDLTSz5gpZxEk5NSJczX040Ph7bi5UHNWLwjngFjFrMxJp8Z\nzr2D4NYfofNoiPrKGs2XFFfyASvlBJqclHICEeE/neow9f5OZGYZBn+2jB9W5XMirosrXPEyXP8V\nxK61zoeKWVPi8SpV0jQ5KeVErWsFMXNkFzpEVObJn/7l8R/Xk5qRlbdiixusaY/EFb7pB+unlHyw\nSpUgTU5KOVlwJU/G39WeUb0b8OPqaK77dBn7jpzKW7FaS+s4VI128Mv9MOcZyDrHtaSUKuM0OSlV\nCri6CI9c0ZBv7mxHbGIKAz5ewp+bDuWt6BsCt/8CHYbDik9g4nWQfLTkA1aqmGlyUqoU6dk4jJkj\nu1An2Jdh363mf3O2kpmVa1YJV3fo9yYM+hT2/2OdD3XoX+cErFQx0eSkVClTs7IPPz7QiVs61OKz\nBbu4/auVxJ/M5yKGrW+Fu2ZbXXtfXQkbfy75YJUqJsWWnETkaxGJE5GNDmVvi8hWEdkgIr+ISKDD\nsqdFZKeIbBORq4orLqXKAi93V16/rgXv3HgZa/Yf4+qPFhO1N5/uuxptrQsYVm0B0+6CuS9Cdj4D\nKpQqY4qz5TQe6Jur7C+guTGmJbAdeBpARJoCQ4Fm9jqfiohrMcamVJlwQ9sa/DqiMz4ergwdu4Iv\nF+/OexFDvypwx0xoexcseR8mDYGUROcErFQRKbbkZIxZBBzNVfanMeb08KIVQA378SBgijEmzRiz\nB9gJtC+u2JQqS5pU82fGyC70ahzGq79vYcSkNSTlvoihmwdc8wEMeB92L4BxvSBuq1PiVaooOPOY\n093AbPtxOHDAYVm0XaaUAvy93Pni9rY83a8xf2w6zMAxS9h+OJ+LGEbeDXf8Bmkn4cs+sPX3kg9W\nqSLglOQkIs8CmcD3F7HuMBGJEpGo+Pj4og9OqVJKRLi/ez2+v7cDJ1IyGTRmKdPXxeStWLuTdRwq\npAFMuQV+fxTSkko6XKUuSYknJxG5ExgA3GrOdp7HADUdqtWwy/Iwxow1xkQaYyJDQ0OLNValSqOO\ndYOZNaoLLcIDGD1lHc9P30haZq5BEAHh1ki+jiNg1VfwWSfYvdA5ASt1EUo0OYlIX+AJYKAxJtlh\n0QxgqIh4ikgE0ADQC9kodQ5h/l58f18HhnWry4Tl+xjyxQpiE3NdxNDdC/q+DnfPARd3mDDQmt08\nLZ/uQKVKmeIcSj4ZWA40EpFoEbkHGAP4AX+JyDoR+RzAGLMJmApsBuYAI4wxOh5WqfNwd3Xhmf5N\n+OzWNuyMS+LqjxazaHs+Xd21OsIDS6DTQxD1NXx6uTVoQqlSTPIMSy1DIiMjTVRUlLPDUMrpdscn\nMXziGrbHneS/fRryUM/6uLhI3or7/4HpD8KRndbQ8ytfAU+/kg9YVVgistoYE1lQPZ0hQqlyoG5o\nJX4ZcTnXtgrnvb+2c/e3qzh2Kp+LE9bqYLWiLh8Jq8fDp51g1/wSj1epgmhyUqqc8PFw472bLuPV\na5uzbOcRBny8hA3R+ZyM6+4NV75qXYLDzQu+uxZ+Gw2pJ0o+aKXOQZOTUuWIiHBbx9r8+EAnAG74\nbDnv/LGN5PR8Lq1Rsz08sBguHwVrJsBnl8OueSUcsVL50+SkVDl0Wc1AZo7sQv8WVRkzfye9313I\nb+tj80595O5tHXe6+0/r8XfXwYxR2opSTqfJSalyKsjXgw+GtmbaA50I8vFg5OS1DB27gi0H80k8\nNdvB/Yuh82hY+511LGrn3JIPWimbjtZTqgLIyjZMWbWfd/7YxvGUDG7vWJv/XtGQQB+PvJWjo+DX\nByFhG7S+Ha56DbwCSj5oVS4VdrSeJielKpDE5HTe+2s7E1fsI8DbnceuasTQdrVwzT3sPCMVFr4J\nSz8Ev2pwzUfQoI9zglbliiYnpdQ5bY49wYu/bWLlnqM0q+7PSwObEVmnct6K0aut86Lit0Lr2+DK\n18A7MG89pQpJz3NSSp1T0+r+/DCsIx/d3JojSenc8Ply/vvDOg6fSM1ZsUZbGLYQujwC6yZZx6J2\n/OWcoFWFoi0npSq45PRMPp2/i7GLduPuKozs3YC7OtfB0y3X9T5jVsOvIyB+C7S6zToWpa0odYG0\nW08pdUH2HTnFKzM3M3dLHBEhvjx/TVN6NgrLWSkzDRb+D5Z8AJXCrGNRDa90TsCqTNJuPaXUBakd\n7MuXd7Tjm7vaIcBd36zinvGr2Jtw6mwlN0/o/TzcOxe8g2DSjfDLcEg55rS4VfmkLSelVB7pmdl8\ns3QPH/29g4wsw71dIxjRsz6+nm5nK2WmwaK3YfF74BsK13wIjfo6L2hVJmi3nlLqksWdSOXN2Vv5\neW0MVf29eLp/YwZeVh0Rh6HnsWutY1Fxm6DlUOj3ptWqUiof2q2nlLpkYf5evDekFT8N70SInwej\np6xjyBcr2BzrMMtE9dbWZeG7PQH//gifdIRts50VsiontOWklCqUrGzD1KgDvP3HNhKT07m1Q20e\nuaIhQb4Os0zEroPpI+DwRmg5BPq+CT75nD+lKizt1lNKFYvjyRm8P3c7E5bvxd/bnceubMTN7R1m\nmchMh8XvwOJ3wScYBnwAjfs7NWZVemhyUkoVq62HTvDijE2s2H2UptX8eWlQM9o5zjJxcL11LOrw\nv9DiJuj3P21FKT3mpJQqXo2r+jP5vo6MuaU1x5LTufHz5YyespZDx+1ZJqpdBvfNgx5Pw6af4ZMO\nsGWmc4NWZYa2nJRSlyw5PZPPFuzii0W7cXMRHupVn3u6RJydZeLgBmum88P/QtNB1vGo2p11hokK\nSLv1lFIlbv+RZF75fTN/bT5MnWAfnr+mKb0aV7EWZmVYx6GWfggZySAuUK0V1O0OEd2gZkfw8HHu\nG1DFTpOTUsppFm6P56XfNrE7/hS9Gofx3ICmRIT4Wgsz06xrRu1ZBHsWQvQqyM4EVw+o0f5ssgpv\nC67uzn0jqshpclJKOVV6ZjbfLtvLh3/vIC0zi3u61GVkr1yzTACkJcH+FbBngZWwDm4ADHhUgtqX\nW4kqojtUaQ4uepi8rNPkpJQqFeJOpPK/Odv4aU00Vfw9efyqxvRtXpVKuZPUaclHYe9iK1HtXghH\ndljl3pUhoqudrHpAcD0QyX8bqtRyenISka+BAUCcMaa5XXYj8CLQBGhvjIlyqP80cA+QBYwyxvxR\n0GtoclKq7Fi97xgvztjEvzHHcXMRWtcKpEv9ULo0CKZljUDcXc/RKjoRa3cB2snqRLRV7h9+tlUV\n0Q0CwkvuzaiLVhqSUzcgCZjgkJyaANnAF8Bjp5OTiDQFJgPtgerAXKChMSbrfK+hyUmpsiU727Bi\n9xGW7Exgyc4E/o05jjFQydONjnWD6VI/mC4NQqkX6ptz/r7TjIGju61jVbsXWi2s5CPWsuD6OZOV\nnlNVKjk9OdlB1AFmnk5ODuULyJmcngYwxrxhP/8DeNEYs/x829fkpFTZlpiczrJddrLakcD+o8kA\nVAvwonP9ELrUD6Fz/RBC/Tzz30B2tjXh7OlW1b6lkJ4ECFRtbieq7taxK89KJffG1DkVNjmdo9O3\nxIUDKxyeR9tleYjIMGAYQK1atYo/MqVUsQn08aB/i2r0b1ENsIaiL9mZwNKdCczdcphpq60uvMZV\n/axk1SCEDhGV8fGw/3W5uEDVFtat0whruHrsWitR7VkIK8fC8jHg4maN/ovobo0GrNHOujaVKrVK\nS8tpDLDCGDPRfv4VMNsYM+1829eWk1LlV1a2YXPsCbsLMJ5Ve4+RnpmNu6vQplYQXexk1SI8ALdz\nHa/KSLFHAtrD1mPXgskGN2+o1dHq/qvb3TrfysU1/22oIlXWWk4xQE2H5zXsMqVUBeXqIrSoEUCL\nGgEM71GP1IwsVu09eqYL8N2/tvPuX9vx83Lj8nrBdrIKpU6wz9njVe7eUK+ndQNIPQ57l1qJas8i\n+Psl+Btw87IuO+8bZt+HWrfTj08v8w2xrlWlowSLXWlJTjOASSLyHtaAiAbASueGpJQqTbzcXena\nIJSuDUKhHxxJSmPZriMs3ZnA4h0J/LHpMADhgd7WsaoGIXSuF0xwJYfuO68Aa4b007OkJ8VZSSp2\nLZyKt54n7rdOEk5OsFpZubm42wkr1E5YDo9zJzafYG2RXaTiHK03GegBhACHgReAo8DHQCiQCKwz\nxlxl138WuBvIBB42xhR4tTLt1lNKARhj2Gcfr1qyI4FluxI4kZoJQNNq/nRpYA2uaFenMt4ehUwW\n2VmQcsxKWKfiICneuj8Vf/Zxkv38VDxkpefdhrhYCSpHSyzMTmahDo/tpObmkXcb5UypGK1X3DQ5\nKaXyk5Vt+DfmuN2qimf1vmNkZBk8XF2IrBN0ZiRg8/CAs9ehuhTGWF2Gp1tfp+LgVMK5E1vGqfy3\n4xVwtgXmXx1CG0FYUwhrAoF1ysUMGZqclFLKlpyeyco9R1m6M4ElO4+w5aB1mfkAb3curxdM+4jK\nVA/0JszPkyr+XoT6eZ77pOCikH7KTlwJOVtgji2xxANwfP/Zddx9ciarsCbWY79qZeoYmCYnpZQ6\nh/iTaSzbZXUBLtmZwMHT16ByEOzrQaidrML8PAnzd3xs3Yf6eZ69LEhxSDsJ8dsgbjPEbTl7n3T4\nbB2vAAh1SFan732Diy+uS6DJSSmlCsEYQ3xSGnEn0og7mcrhE9bjwydTz5TFnUgjPimNrOy8/y+D\nfNzPtLbC/Lyo4u95pgUW5m+Vhfp54uVehEks+WjOZHX6cWri2Tq+YXkTVmgj8PIvujgugiYnpZQq\nQlnZhqOn0s8kqzOJzH5++GQa8SdSiTuZRmY+SSzA291OXGeTluPzKvb9RScxY+DkobMJK/500tqa\n8xhXQC0Ia5wzcYU0tIbdlwBNTkop5QTZ2YZjyelnE9fJNOLspHX4xOnn1rKMrLz/f/293Ajz96Kq\nvxc1K/tQs7I3tSr7UDPIh1qVfQj0cc9/3sFzB2Qdu8rd0krYfnaEobhA5bo5E1ZoE2vm9yK+ppYm\nJ6WUKsWMMRxLzjjb8jqRM5HFHk8l+mgyR07lHKJeydPNSlpB3tSsbCWs0wmsRpBP4VteWRnWJLqO\n3YJxW+DorrPnd7m4W60qxwEYjfpd0gAMTU5KKVUOnErL5MCxZA4cTWH/0WQOnL4dS2b/0WRSM3Ke\nKBzm53k2adkJ7PTzKv5eBQ+dz0i1WlWOCSt+i3Vysm8oPL7zkt5PWZu+SCmlVD58Pd1oXNWfxlXz\nDmQwxpCQlM7+o8lEH0tm/5GzSWvlnqNMX5eC4+Evd1ehRpAPNYLsrsIzScy6D/BxB3cvqNbSujlK\nO2ldW6uEaHJSSqkySkQItYe0t60dlGd5emY2B49bLS6r1ZVyptX1+78HSUzOyFHfz8vtbLIKztny\nCg/0wSu0UUm9NU1OSilVXnm4uVA72Jfawb75Lj+RmmF3E6bk6CrcEXeSedviSM8822UoArUr+zD/\nsR4XNiDjImlyUkqpCsrfy51m1QNoVj0gz7LsbOv8r9PHuU4f3yqJxASanJRSSuXDxUWo4u9FFX8v\n2tUp+Uvel/1ZBJVSSpU7mpyUUkqVOpqclFJKlTqanJRSSpU6mpyUUkqVOpqclFJKlTqanJRSSpU6\nmpyUUkqVOmV6VnIRiQf2XeJmQoCEIginvNP9VDi6nwqm+6hwyut+qm2MCS2oUplOTkVBRKIKM317\nRaf7qXB0PxVM91HhVPT9pN16SimlSh1NTkoppUodTU4w1tkBlBG6nwpH91PBdB8VToXeTxX+mJNS\nSqnSR1tOSimlSh1NTkoppUqdCpucRKSviGwTkZ0i8pSz4ymNRKSmiMwXkc0isklERjs7ptJMRFxF\nZK2IzHR2LKWViASKyDQR2SoiW0Skk7NjKo1E5L/239xGEZksIl7OjqmkVcjkJCKuwCdAP6ApcLOI\nNHVuVKVSJvCoMaYp0BEYofvpvEYDW5wdRCn3ITDHGNMYuAzdX3mISDgwCog0xjQHXIGhzo2q5FXI\n5AS0B3YaY3YbY9KBKcAgJ8dU6hhjDhpj1tiPT2L9Iwl3blSlk8j/t3dvIVaVYRjH/09NwcwE0YEg\nG8mhLLpyzKTDRIIGEYV0kYRkkBAhVNBVZITUpXSgiyCC6QTNRWVCQUGCZVnEjFgjokEXHXTMcAgs\nCkObni7WmpgG0Qb23uuT/fzu9jfrW/tdMDPPXof9fhoA7gBGmq6lVJLOB24BXgGwfdz20WarKlYP\n0CupB+gDfmq4no7r1suurngAAANNSURBVHC6DDg46/Uk+ad7SpIWAUuBsWYrKdYLwGPA300XUrBB\nYAp4rb78OSKpv+miSmP7EPAscAA4DPxqe1uzVXVet4ZTzIOk84B3gUdt/9Z0PaWRdCdwxPbupmsp\nXA9wLfCS7aXAH0Du984h6QKqKzmDwAKgX9K6ZqvqvG4Np0PAwlmvB+qxmEPSOVTBNGp7a9P1FGoY\nWC3pB6pLxCslvdlsSUWaBCZtz5x9b6EKq/ivW4HvbU/ZPgFsBW5quKaO69Zw2gUsljQo6Vyqm43v\nN1xTcSSJ6v7AN7afb7qeUtneaHvA9iKq36WPbXfdJ93Tsf0zcFDS1fXQKmB/gyWV6gBwg6S++m9w\nFV344EhP0wU0wfZfkh4GPqJ6EuZV2/saLqtEw8B9wF5JE/XYE7Y/bLCmOLM9AozWHwq/A9Y3XE9x\nbI9J2gJ8RfXE7Nd0YSujtC+KiIjidOtlvYiIKFjCKSIiipNwioiI4iScIiKiOAmniIgoTsIposUk\nTUuaqDtKvyOpb57zR+bTYFfS/ZJenH+lEeVKOEW03jHbQ3VH6ePAhv87UdLZth+wnS+nRldLOEW0\n107gSgBJ6ySN12dVL9dLtyDpd0nPSdoD3Chph6Tr6p+tlbS3PgvbPLNTSeslfStpnOrL0jPja+pt\n90j6rKNHGtFCCaeINqmXO7idqsPGNcA9wLDtIWAauLfetB8Ys73E9uez5i8ANgMrgSFguaS7JF0K\nPE0VSjdTrUk2YxNwm+0lwOq2HmBEG3Vl+6KINuud1e5pJ1V/wgeBZcCuql0avcCReptpqua6cy0H\ndtieApA0SrUeEnPG3wKuqse/AF6X9DZVw9CIM1LCKaL1jtVnR/+qG3i+YXvjSbb/0/Z0K97Y9gZJ\n11MtfLhb0jLbv7Ri3xGdlMt6EZ2xHbhb0iUAki6UdPlp5owDKyRdXN+fWgt8SrXg4wpJF9VLmqyZ\nmSDpCttjtjdRLey38GQ7jihdzpwiOsD2fklPAtsknQWcAB4CfjzFnMOSHgc+AQR8YPs9AElPAV8C\nR4GJWdOekbS43n47sKcNhxPRdulKHhERxcllvYiIKE7CKSIiipNwioiI4iScIiKiOAmniIgoTsIp\nIiKKk3CKiIji/AN7qM9MDsJlwQAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "c6diezCSeH4Y",
        "colab_type": "text",
        "slideshow": {
          "slide_type": "slide"
        }
      },
      "source": [
        " ## 任务 2：用测试数据进行评估\n",
        "\n",
        "**确认您的验证效果结果经受得住测试数据的检验。**\n",
        "\n",
        "获得满意的模型后，用测试数据评估该模型，以与验证效果进行比较。\n",
        "\n",
        "提示：测试数据集位于[此处](https://download.mlcc.google.cn/mledu-datasets/california_housing_test.csv)。"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "icEJIl5Vp51r",
        "colab_type": "code",
        "cellView": "both",
        "colab": {
          "test": {
            "output": "ignore",
            "timeout": 600
          }
        }
      },
      "source": [
        "california_housing_test_data = pd.read_csv(\"https://download.mlcc.google.cn/mledu-datasets/california_housing_test.csv\", sep=\",\")\n",
        "\n",
        "# YOUR CODE HERE"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "vvT2jDWjrKew",
        "colab_type": "text"
      },
      "source": [
        " ### 解决方案\n",
        "\n",
        "点击下方即可查看可能的解决方案。"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "FyDh7Qy6rQb0",
        "colab_type": "text"
      },
      "source": [
        " 与顶部代码类似，我们只需加载合适的数据文件、对其进行预处理并调用预测和 mean_squared_error 即可。\n",
        "\n",
        "请注意，由于我们会使用所有记录，因此无需对测试数据进行随机化处理。"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "vhb0CtdvrWZx",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        },
        "outputId": "bb289d5f-5e7b-4b7c-9147-fb295333c725"
      },
      "source": [
        "california_housing_test_data = pd.read_csv(\"https://download.mlcc.google.cn/mledu-datasets/california_housing_test.csv\", sep=\",\")\n",
        "\n",
        "test_examples = preprocess_features(california_housing_test_data)\n",
        "test_targets = preprocess_targets(california_housing_test_data)\n",
        "\n",
        "predict_testing_input_fn = lambda: my_input_fn(test_examples, \n",
        "                                               test_targets[\"median_house_value\"], \n",
        "                                               num_epochs=1, \n",
        "                                               shuffle=False)\n",
        "\n",
        "test_predictions = dnn_regressor.predict(input_fn=predict_testing_input_fn)\n",
        "test_predictions = np.array([item['predictions'][0] for item in test_predictions])\n",
        "\n",
        "root_mean_squared_error = math.sqrt(\n",
        "    metrics.mean_squared_error(test_predictions, test_targets))\n",
        "\n",
        "print(\"Final RMSE (on test data): %0.2f\" % root_mean_squared_error)"
      ],
      "execution_count": 8,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Final RMSE (on test data): 103.34\n"
          ],
          "name": "stdout"
        }
      ]
    }
  ]
}